1. Technical Field
The present invention relates to a distributed processing technique in grid computing in which a master computer (hereinafter referred to as “master”) makes a plurality of worker computers (hereinafter referred to as “worker”) execute distributed processing as a processing group.
2. Description of the Related Art
In the flow of a job taken between a master and a worker which can communicate with each other through a network, the master inputs a job and data required to process the job to the worker. Subsequently, the worker to which the job is input executes the processing of the job. Then, the worker returns the processing result of the job to the master. The master executes the above operation on plural workers, thereby making the workers execute the distributed processing of the overall job.
In general, the job input to the master is estimated to be a job the execution time of which is sufficiently larger than the communication delay time between the master and the worker (coarse-grained job). A technique of subjecting coarse-grained jobs to distributed processing has been in practical use. For example, System Walker CyberGrip (registered trademark) is a known grid middleware for executing distributed processing on a large amount of batch jobs, etc.
Furthermore, Japanese Laid-open Patent Publication No. 2004-110318 describes a technique of allocating processing on a task group basis, each task group comprising plural tasks. Specifically, first, the processing of a task group is allocated from a higher managing device to a lower managing device. Next, tasks contained in the task group are allocated from a lower managing device to a task execution device, and the task execution device executes the task and transmits the execution result to the lower managing device. The lower managing device collects the execution result and transmits the collection result to the higher managing device. Finally, the higher managing device aggregates and outputs the collection results.
Furthermore, Japanese Laid-open Patent Publication No. H11-195007 describes a technique of dynamically adjusting the task amount allocated from the master to the worker in accordance with the task request from the worker to the master. Specifically, the master first allocates a task to be processed to each worker. At that time, the same task load is first allocated to each worker, and the task load to be allocated in response to one task request is varied in accordance with a task request frequency from each worker within a predetermined time.
In the related techniques described above, the jobs input to the master are input to the workers on a job basis, thereby making the workers execute the distributed processing on all the jobs. This is an effective method in a case where the execution time of each job is sufficiently larger than the communication time between the master/worker. However, there are times when execution time of each job is shorter than the communication time between the master and the worker (fine-grained-job).
When jobs having a short execution time are input to workers on a job basis, overheads such as the communication time between the master and the worker, the idle time of a worker, etc. in the job processing increases significantly. Moreover, this induces a problem that the traffic on the network is increased and thus the required time taken for job processing is increased.
The present invention has an object to solve the above problem of the related arts, and has an object to reduce the communication traffic between a master and a worker in job processing, and implement efficient distributed processing by properly allocating a job group containing a bundle of jobs having short execution time.